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Abstract 

Advanced mobile networking technology applicable to 
mobile sensor platforms was developed, deployed and 
demonstrated. A two-tier sensorweb design was developed. 
The first tier utilized mobile network technology to provide 
mobility. The second tier, which sits above the first tier, 
utilizes 6LowPAN (Internet Protocol version 6 Low Power 
Wireless Personal Area Networks) sensors. The entire network 
was IPv6 enabled. Successful mobile sensorweb system field 
tests took place in late August and early September of 2009. 
The entire network utilized IPv6 and was monitored and 
controlled using a remote Web browser via IPv6 technology. 

This paper describes the mobile networking and 6LowPAN 
sensorweb design, implementation, deployment and testing as 
well as wireless systems and network monitoring software 
developed to support testing and validation. 

1.0 Introduction 

The United States National Aeronautic and Space 
Administration (NASA) Earth Science Technology Office 
(ESTO) manages the development of advanced technologies 
and applications that are needed for cost-effective missions. 
ESTO plays a major role in shaping Earth science research 
and application programs of the future, aggressively pursuing 
promising scientific and engineering concepts, and ensuring 
that the program maintains an effective balance of investments 
in order to advance technology development. ESTO sponsored 
NASA’s Glenn Research Center (GRC) to research and 
deploy advanced mobile networking technology applicable to 
mobile sensor platforms. As a result, GRC personnel 
developed a two-tier sensorweb design. The first tier utilized 
mobile ad hoc network (MANET) technology to provide 
mobility. The second tier, which sits above the first tier, 
utilized 6LowPAN (Internet Protocol version 6 Low Power 
Wireless Personal Area Networks) sensors. The entire network 
was IPv6 enabled. 

This paper describes the mobile networking and 6LowPAN 
sensorweb design, implementation, deployment and testing as 
well as wireless systems and network monitoring software 
developed to support test and validation. In addition, issue in 
deployment and idiosyncrasies of various technologies and 
tools are identified and discussed. 


2.0 Mobile Networking 

GRC collaborated with Cisco System, Incorporated to 
research and deploy advanced mobile networking technology 
applicable to mobile sensor platforms. GRC personnel 
developed a two-tier sensorweb design. The first tier utilized 
mobile network technology from Cisco Systems to provide 
mobility. GRC utilized a combination of mobile IP and mobile 
ad hoc networking alpha software from Cisco System, known 
internally to Cisco as “Duetto”. 1 Duetto covered implementation 
of the mobile router functionality for Mobile IPv6 and nested 
NEMO (NEtworks in MOtion) support. In addition. Duetto 
performed full Tree Discovery and MANET (Mobile Ad hoc 
NET work) support called “Bubbles”. Portions of the “Bubbles” 
protocol has applications in low power sensorwebs for smart 
buildings, industrial controls and monitoring and has been 
incorporated into specifications being developed by the Internet 
Engineering Task Force working group on Routing Over Low 
power and Lossy networks (roll) (Ref. 1). 

Some of the major features of this software build include: 
tree discovery, nemo support, and Reverse Routing Header 
(RRH). Tree Discovery establishes a tree using extended IPv6 
Neighbor Discovery. Neighbor Discovery occurs at the speed 
of the link layer (L2); therefore, the tree discovery occurs very 
quickly. The nemo portion of the Duetto code exploits the tree 
to optimally get out of a nested set of Mobile Routers (MRs) 
and register to the mobile-ip Home Agent. RRH extends the 
nemo support to provide route optimization and added 
security. The “Bubbles” or MANET portion of the Duetto 
code also extends Neighbor discovery in order to quickly 
establish the routes down the cluster. Finally, since “Bubbles” 
and the nested mobile networks (nested nemo) both exploit the 
same cluster (tree), switching back and forth from Mobile IP 
(nemo) to ad-hoc “Bubbles” is optimized. 

The Duetto software was originally debugged by Cisco 
Systems personnel using a wired network. GRC personnel 
implemented a wireless network with radios on each interface. 


'This code was developed by Cisco System France. Primary 
designers include Pascal Thubert and Marco Molteni. The code was 
Cisco IOS Software, 3200 Software (C3250-ADVENTERPRISEK9- 
M), Experimental Version 12.4(20060331:114112) [pthubert-clairette 
168] 
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Numerous problems were uncovered in the wireless 
network that did not show up in the wired network. These 
problems were rectified by either code fixes performed by 
Cisco, network configurations or readdressing the network — 
whichever was the appropriate corrective action. 

The Duetto code was installed on Cisco 3200 series Mobile 
Access Routers Cards (MAR), which utilize the PC/1 04-Plus 2 
hardware standard. In addition to the router, a Fast Ethernet 
Mobile Switched Interface Card (FESMIC) was used as were 
two Cisco C3201 Wireless Mobile Interface Cards (WMIC). 3 
A WMIC is connected to the ingress interface (FESMIC, 
vlanl) and its station-role set to “root bridge” mode making 
this a “Parent” radio. A separate WMIC radio was connected 
to the egress or wan interface (Fa-0/0) and station-role was set 
to “workgroup-bridge” making this a “Child” radio. The “root 
bridge”/“working group” pair was the only mode found to 
work with IPv6. 4 Other Combinations broke the IPv6 stateless 
auto-configuration, which is necessary for the Duetto protocol. 
In addition the switch ports (FESMIC ports) on the MAR had 
to have spanning-tree PortFast set. Spanning-tree PortFast 
causes a port to enter the spanning-tree forwarding state 
immediately, bypassing the listening and learning states. 

Figure 1 shows a typical mobile node. The egress interface 
is connected to a wireless radio configured in “Child” mode. 
This radio can connect to a single “Parent” radio to form a tree 
structure. Likewise, the Ingress interface is connected to a 
radio configured to be a “Parent”. Note: many mobile nodes 
can connect to a single “Parent” as a single “Parent” can 
simultaneously associate with multiple Children. 

In a wired network, the ingress and egress inputs never 
connect to each other and thus never see each other. In a 
wireless system, the MANET radio, 5 which resides on the 
egress port, can see a MANET radio attached to its own 
ingress port. Cisco implemented a loop avoidance algorithm in 
the Tree Discovery process to ensure that the mobile 
networking software prohibited a single node’s egress port(s) 
from connecting to its ingress port as well as ensure that a Top 
Level Mobile Router (TLMR) 6 did not connect to a node that 
is deeper in its own tree. Even so, we still encountered 
problems with layer-2 radio loops. Each mobile network pair 
had to be configured to ensure that the “Child” radio was 

'Due to the combined power requirements exceeding that of the 
power supply (Datel MAPC-104) and the PC/104-Plus standards, the 
WMICs were in a separated PC- 104 stack. 

3 The Cisco 3201 WMIC is a 2.4 GHz, 802.11b.g radio. 

4 WMIC cards are not capable of full MANET radio functionality. 
Bridge-Mode was the only combination found to work. An exhausted 
amount of testing of various configurations was performed. However, 
there were several issues that were resolved during testing (i.e. 
spanning tree, RFI, etc.). There may be other configurations that will 
work with IPv6, however, this is one that configuration that definitely 
will work 

5 A true MANET radio will listen and communicate with all other 
MANET radios. 

‘’TLMR refers to the root router of the topology tree that is formed. 
For example, router 509 in figure 8 is the TLMR of that tree. 
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Figure 1. — IPv6 mobile node with radio. 

prohibited from associating with the “Parent” radio on the 
same mobile node. It is possible for two mobile routers to 
become isolated because the “Parent” radio of router A 
attaches to the “Child” radio of router B and the “Parent” radio 
of router B attaches to the “Child” radio of router A. 7 This is a 
legitimate configuration — particularly if only two systems 
exist. The Duetto code will make one of those two nodes the 
TLMR. However, those two systems become isolated from the 
rest of the group. In order to solve this problem, one would 
have to have some interaction between the radio system and 
the router (layer-2/layer3 interaction) such that if the router 
realized that a single layer loop has occurred, then, the layer-3 
routing software would notify the radio whereupon that radio 
would then to try and reassociate with a different radio. 

For demonstration purposes, we could configure the radios 
to prevent specific parent/child pairs from associating with 
each other to avoid routing loops caused by layer-2 radio 
associations. This helped demonstrate the layer-3 mobile 
networking Duetto code, but is not a scalable solution. The 
WMICs also appeared to have a feature whereby one could 
prioritize the Service Set Identifications (SSIDs). However, 
we could not get that feature to work in both “Parent” and 
“Child” radios. Furthermore, even if this would have enabled 
us to prevent some isolation loops, it is also not a scalable 
solution and requires a priori knowledge of the network and 
possible contacts between nodes. 

Another problem that was uncovered was that the initial 
configurations showed proper route tables in the MAR routers 
and that all routes appearing to be accessible. However, during 
testing, it was apparent that not all nodes were truly reachable 
even though the route tables indicated otherwise. 
Troubleshooting the system uncovered the following: 

1. The radios were turned on and “Child” nodes would 
associate to a parent. 

2. Initially the egress (fa0/0) interface would get a IPv6 
address from a Routing Advertisement (RA) with a 
default lifetime of 1800 sec. 

7 This scenario did not occur often when the MANET nodes were 
setup in the lab, but is was a common occurrence during the 
migration to the outdoors deployment. 
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3. Spanning tree would kick in and block the connection. 

4. Bubbles would think it had a valid address and maintain 
an entry in the routing table, even though the connection 
was blocked. 

5. 30 min later the RA would expire, spanning tree was still 
blocking so the entry would be deleted from the routing 
table. 

The solution was to do the following in the router 
configurations: 

1 . RA lifetime was reduced to 10 sec 

2. Spanning-tree portfast was enabled on the switching 
interfaces of the routers. 

3. Layer-2 spanning-tree was disabled in the WMICs. 
(Although not documented as a fix to this problem, the 
command is in the WMIC’s configuration file.) 

An interesting feature of the Duetto code is that the network 
and tree discovery are performed using IPv6 features. Once a 
tree is established using IPv6, the code propagates IPv4 routes. 
The code forwards its routing table out its egress interface to 
the next highest-level router in the tree (this includes the 
bubbles entries along with any subnets that are defined on its 
interfaces). The code also passes the IPv4 routing table 
thereby creating an IPv4 MANET in addition to the IPv6 
MANET. This was discovered by accident, but proved 
extremely useful. The Cisco PC 104 Wi-Fi cards, WMICs, are 
not IPv6 capable so IPv4 address were assigned for 
administrative purposes. One could then access a router via 
IPv6 and then telnet from the router to the locally attached 
WMIC via IPv4. It was discovered after the bubbles network 
was functional that the WMICs could be accessed from any 
node on the system using IPv4 addressing. 

2.1 Routing Nuances 

The “Bubbles” portion of the Duetto code uses auto 
configuration to create the tree structure. Each egress interface 
is set to auto configure and is connected to a radio in “child” 
mode. Each ingress interface has a fixed /64 address and sends 
a route advertisement out. Each ingress interface is also 
connected to a “Parent” radio. When a “Child” radio attaches 
to a “Parent” radio it receives a new RA and becomes part of 
the Parent’s network. The IPv6 auto configuration on the IOS 
version used would only work if the subnet of the advertising 
interface was a / 64. Therefore each egress IPv6 interface 
needed to be configured with a /64 subnet. 

It is important to note that the Duetto code only forwards 
subnets assigned to its interfaces and any subnet entries 
received from routers farther down the tree. The Duetto code 
does not propagate static routes (static route redistribution) or 
subnets assigned to tunnels. 
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2.2 Routing to the 6LowPAN Sensors 

The 6LowPAN network is described in the next section, 
Section 3.0, IPv6 Sensorweb. However, the routing is described 
here. 

The 6LowPAN network consists of a server, 6LowPAN 
routers and 6LowPAN sensors. The routers and server were 
attached to Ethernet interfaces on the mobile ad hoc routers, the 
Cisco MAR routers (Fig. 1). In the case of the 6LowPAN 
router, it has two interfaces, an egress and ingress interface. The 
egress interface of the 6LowPAN router is connected to the 
ingress interface of the ad hoc router subnetwork. The ingress 
interface of the 6LowPAN router is connected to the 6LowPAN 
sensors — often via the 802.15.4 wireless link. The ad hoc router 
must provide address space to accommodate at least two 
subnetworks: one for the 6LowPAN egress interface and one for 
the 6LowPAN ingress interface. In order to accomplish this, a 
secondary address is implemented on the ad hoc router ingress 
interface providing a /63 subnet for the 6LowPAN router and 
sensors. This subnet “effectively”, not literally, is split into two 
/64 subnets with one /64 subnet allocated to the 6LowPAN 
sensors. However, in order for the ad hoc router to reach the /64 
subnet of the 6LowPAN network, a static route is also needed. 
The following is from an actual configuration: 

interface Vlanl 

description "INGRESS or ATTACHMENT INTERFACE" 
ip address 10.50.13.1 255.255.255.0 
ipv6 address FDAD:9F5B:4B7D:500D::l/64 
ipv6 address FDAD:9F5B:4B7D:50D0::l/63 
ipv6 route FDAD:9F5B:4B7D:50Dl::/64 
FDAD:9F5B:4B7D:50D0::77 

Note: the 10.50.13.0/24 IPv4 subnet actually gets propagated up 
the tree and will show up in the routing table. 

• FDAD:9F5B:4B7D:500D::/64 is the subnet that other ad 
hoc routers attach to via their “Child” radio. This is the 
subnet that is advertised using router advertisements. 

• FDAD:9F5B:4B7D:50D0::/63 is an additional subnetwork 
that is propagated up the tree. The secondary interface 
address on the ad hoc router ingress interface is 
FDAD:9F5B:4B7D:50D0::1. For consistency, the egress 
interface of the 6LowPAN router was always given an 
address in the lower portion of the /63 subnet with the last 
64 bits as 77. In this example, the 6LowPAN router’s egress 
interface is FDAD:9F5B:4B7D:50D0::77. The 6LowPAN 
router is configure to have its ingress interface use the upper 
half of the /63 subnet or FDAD:9F5B:4B7D:50Dl::/64. The 
route statement in the configuration enables the ad hoc 
router to understand where to send data destined to the 
6LowPAN sensors. 8 

8 A static address has to be used for the following reasons: Since the 
IPv6 subnet was a /63, the router would not support auto-configuration. 
The address had to be known so an IPv6 static route could be pre- 
configured. Also, the 6LowPAN router was administered via HTTP so 
it had to be at a known address. 
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Figure 2. — “Bubbles” IPv6 mobile ad hoc network. 


The seven node IPv6 mobile ad hoc network is shown in 
Figure 2. One node is connected to the Internet to provide 
connectivity to the general Internet. This connectivity was via 
and IPv4 network while the mobile IPv6 mobile ad hoc 
network was configured to use IPv6 Unique Local Addressing 
to ensure that no experimental traffic would leak to the open 
Internet. Each mobile node had one “Parent” radio and one 
“Child” radio. Thus any mobile node could become the top- 
level mobile router (TLMR) depending on the movement and 
the RF connectivity of the devices. For our test network, the 
two extreme limits for forming tree structures are: 1) a single 
serial string of 7 routers (i.e., 6 layers deep); or 2) one top- 
level router with 6 children (i.e., 1 layer deep). 

3.0 IPv6 Sensorweb 

In order to demonstrate a mobile sensorweb, a search was 
performed to identity potential IPv6 sensors. Numerous 
6LowPAN sensors were identified. ArchRock IPv6 sensors 
(Ref. 2) were procured and integrated into the mobile ad hoc 
network. These sensors were configured to be within the 
address space of the mobile ad hoc network forming a second 
tier network sitting above the first tier. The entire nemo 
mobile IP network and “Bubbles” ad hoc network along with 
the sensor network were all connective via IPv6 addressing. 

The 6LowPAN sensor network that was constructed 
consisted of a server, 3 6LowPAN routers and numerous 
6LowPAN sensors. The server manages interconnected 
collections of IP -based wireless sensor networks using a 
common web services architecture and web browser interface. 
The server was located in the Protocol Research Lab 
(bldg. 54) and connected to the 6LowPAN routers via the 
Cisco IPv6 MANET network. The routers are IP-based 
802.15.4 wireless sensor networking devices that connect 
6LowPAN mesh networks to the main server via Wi-Fi and 
Ethernet interfaces. In our setup, the 802.15.4 wireless links 
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Figure 3. — Wireless personal area networks sensor 
(time scale 24 hr from midnight to midnight in 6 hr 
increments). 

connected the 6LowPAN sensors to the router and an Ethernet 
interface was connected to the “Bubbles” MANET router. The 
ArchRock Wi-Fi link was not used. 

The ArchRock server keeps track of all 6LowPAN routers 
and nodes and maintains status and history of the sensors, with 
the exception of the ArchRock IP serial Nodes. A Linux server 
that also provided the topology drawings queried the IP serial 
nodes. This Linux server was co-located with the 6LowPAN 
server. 

The ArchRock server standard configuration uses virtual 
private networks (VPNs) to connect to the 6LowPAN routers. 
There was a bug in the ArchRock server code relative to the 
use of VPNs over IPv6. In order to keep everything in the 
network on IPv6 address space, VPNs were abandoned (IPv4 
addresses were configured, but only for debugging the VPN). 
The routing configuration for this has previously been 
described in Section 2.0, Mobile Networking. 

Sensors for temperature, power, humidity and luminance 
were used in this testbed for demonstration purposes. These 
sensors also monitored the voltage of the battery pack that 
powered the transportable MANET nodes. These 6LowPAN 
sensors were selected because they provided some data that 
could easily be correlated to the conditions at hand. 

One sensor group was located in a cargo van that we could 
drive throughout Glenn Research Center. Figure 3 shows the 
detail readout of that particular sensor. The sensor shown is 
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Figure 4. — Panasonic IPv6 WebCAM (rainy day at 
Glenn Research Center). 


500C-8E, which was located inside the cargo compartment of 
the mobile van. From the sensor readings one can determine 
that the lights were off from 6:00 p.m. to about 9:00 a.m. 
(second reading from the bottom) and the temperature 
increased soon after the Sun came up at approximately 
9:00 a.m. (bottom reading). 

In addition to the 6LowPAN sensors, an IPv6 enabled 
Webcam was installed on the van. A Panasonic Webcam was 
used. One undocumented nuance with this particular device 
was that the camera had to be configured with an IPv4 address 
first, then one could configure the IPv6 address. Furthermore, 
this had to be done in the first 10 to 20 min of startup or the 
camera locked out changes of its configuration, for security 
reasons. 

Figure 4 shows the output of the IPv6 Webcam attached to 
Mobile Node 500C, the mobile VAN. This Webcam was 
controlled using IPv6. 
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4.0 Field Tests 

The mobile sensorweb system field tests took place in late 
August and early September of 2009. Figure 5 shows a 
conceptual field deployment. Here, we only needed to have 
one mobile unit in order to make the entire network topology 
change. Hence “network mobility” was exercised at each 
mobile router due to changes in network topology as the van 
moved about the GRC campus. In order to remotely monitor 
each mobile node, we configured the network such that at 
certain points in the movement of van, every node was 
reachable. This was not necessary for the network to function. 
Rather, this was done as a matter of convenience for testing 
and debugging. 



Figure 5. — Mobile network notional test deployment. 



Figure 6. — MANET router locations. 


Figure 6 shows the locations of the seven MANET router 
nodes place throughout NASA’s Glenn Research Center. The 
“Star” indicates the van. As the van moves around the lab, the 
wireless connectivity changes and the MANET topology 
changes. Two MANET routers were co-located in the Protocol 
Lab (bldg. 54). One of these routers (router 5009) had external 
antennas mounted on the roof of the building in order to 
connect with the other wireless systems scattered throughout 
the GRC campus. It is important to note, not all nodes could 
communicate directly with each other. In order to connect 
nodes that did not have direct RF connectivity, these nodes 
had to hop through others. This was done intentionally to 
exercise the hill capabilities of the Duetto code. 

In order to monitor the mobile nodes, GRC developed a 
simple network management system, which showed location 
of the nodes and simple status. That system consisted of two 
independent pieces: a location system, which used GPS 
(Global Positioning System); and, a network monitoring 
system, which was implemented using PERL (Practical 
Extraction and Report Language). 
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Figure 7. — GPS location of 500C Van Node (IPv6). 


The first piece utilized GPS and the 6LowPAN sensorweb 
serial devices to obtain GPS data over the network. A Linux 
computer polled the GPS device and utilizes Google Maps to 
map the location of the GPS receiver onto a map of Glenn 
Research Center. The GPS receiver was placed in the van to 
show the location of the van as it travels around the GRC 
campus. This provided a reasonable visual check to see if 
network topology changes match what would be expected 
from the RF link possibilities at any given instant in time. In 
Figure 7, the location of the van, mobile node 500C, is 
depicted by the red dot in the middle of the map. 

4.1 Tracking and Monitoring Tool 

A PERL based network monitoring tool was developed and 
used to display the network topology of the 7-node network. 
The PERL script used Graphvis (Ref. 3) to display the network 
topology. Figures 8 and 9 show two configurations of the same 
seven-node network along with the 6LowPAN sensors. The 
6LowPAN routers (light blue boxes) are labeled “PhyNet 
Router” and the 6LowPAN sensors are labeled “IP sensor”. The 
IPv6 mobile ad hoc (MANET) routers use Unique Local 
Address (ULA) to ensure testbed routes do not propagate to the 
Internet. That address space is FDAD:9F5B:4B7D::/48. We 
further subnet this space down to /64 addresses for the mobile 
ad hoc network routers and /63 subnetworks for each 
6LowPAN router and its associated sensors — see “Routing to 
the 6LowPAN Sensors” in Section 2.0, Mobile Networking. 
The routers are labeled according to the last 16 bits of their /64 
subnet. For example: 500A Router is the router on subnet 
FDAD:9F5B:4B7D:500A. By our own convention, the /63 


subnet is associated with an ad hoc router that has a 
6LowPAN network attached is FDAD:9F5B:4B7D:50X0/63 
where “X” is a alpha character associated with the ad hoc 
ingress /64 subnet. All Vlanl interface addresses are: 50X0:: 1 
and all PhyNet router addresses are: 50X0::77. The Webcam 
is located in the van node on subnet: 500C::/64. 

The node labeled “5009” is located in our protocol lab 
(bldg. 54) and physically never moves. It is attached to the 
open Internet. The 6LowPAN server is attached to this router. 
The router labeled “500C” is the van. As the van moves about 
the GRC campus, the network topology changes, often leaving 
some nodes isolated. Note: changes in network topology can 
occur due to movement of nodes or simply changes in RF link 
characteristics due to blockage, atmospheric conditions, 
interference, or other factors. 

Figure 8 shows a mobile ad hoc network with a tree depth 
of three. Note that the mobile router “500E” at building 142 is 
isolated. No other systems are in contact with this route as it 
was intentionally positioned to only become connected to the 
entire network if the mobile node (Van) came close enough to 
close the RF links. Also, in Figure 8, the top-level router is the 
router located in building 54 and connected directly to the 
Internet. 

Figure 9 shows a change in topology from Figure 8. The ad 
hoc router at building 142 is now connected to the rest of the 
network via the ad hoc router in the van. The tree depth here is 
4 and the top-level ad hoc router is the building 54 lab router, 
which has a 6LowPAN router attached. This 6LowPAN router 
has two 6LowPAN sensors attached to it. Only mobile ad hoc 
routers with 6LowPAN servers or routers required additional 
network address space — see Section 2.0 subsection “Routing 
to the 6LowPAN Sensors”. 


5.0 Sensorweb Discovery 

Web services and sensor discovery were not implemented 
in this demonstration. However, since one MANET router, the 
router located in the Protocol Lab, was connected to the 
Internet and since the ArchRock server was also tied to this 
MANET router, the entire system was reachable via the Web. 
Furthermore, since this router node was always connected to 
the same Internet address, we would have easily registered 
that address in the Internet Domain Name System (DNS). 
Thus, one should be able to access the mobile sensorweb 
information using the Open Geospatial Consortium (OGC) 
specifications and standards (Ref. 4) so long as security 
accesses privileges were granted. 
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Figure 8. — Mobile network topology 1. 
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Figure 9. — Mobile network topology 2. 
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6.0 Conclusions 

A two-tier sensor network was successfully demonstrated 
that utilized an ad hoc mobile network to handle mobility and 
a 6LowPAN sensor network to provide sensor readings. The 
entire system operated using IPv6 technology. In addition, and 
IPv4 mobile network could be constructed but only if IPv6 
was operational as IPv6 neighbor discovery is used to discover 
and construct the ad hoc network. 

A parent/child system was used in this network. Such a 
system could result in mobile ad hoc router pairs becoming 
isolated. Therefore, some interaction between the layer-3 
mobile ad hoc networking code and the radio system should 
be considered to help alleviate this problem. 

There continues to be a need for a radio that is developed 
specifically to work with layer-3 ad hoc and mobile 
networking rather than in conflict. 
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